How Do Learners Receive Feedback on Note-Writing? A Scoping Review.

PURPOSE: The literature assessing the process of note-writing based on gathered information is scant. This scoping review investigates methods of providing feedback on learners' note-writing abilities. METHOD: Scopus and Web of Science were searched for studies that investigated feedback on student notes or reviewed notes written on an information or data-gathering activity in health care and other fields in August 2022. Of 426 articles screened, 23 met the inclusion criteria. Data were extracted on the article title, publication year, study location, study aim, study design, number of participants, participant demographics, level of education, type of note written, field of study, form of feedback given, source of the feedback, and student or participant rating of feedback method from the included articles. Then possible themes were identified and a final consensus-based thematic analysis performed. RESULTS: Themes identified in the 23 included articles were as follows: (1) Learners found faculty and peer feedback beneficial; (2) Direct written comments and evaluation tools, such as rubrics or checklists, were the most common feedback methods; (3) Reports on notes in real clinical settings were limited (simulated clinical scenarios in preclinical curriculum were the most studied); (4) Feedback providers and recipients benefit from having prior training on providing and receiving feedback; (5) Sequential or iterative feedback was beneficial for learners but can be time intensive for faculty and confounded by maturation effects; and (6) Use of technology and validated assessment tools facilitate the feedback process through ease of communication and improved organization. CONCLUSIONS: The various factors influencing impact and perception of feedback include the source, structure, setting, use of technology, and amount of feedback provided. As the utility of note-writing in health care expands, studies are needed to clarify the value of note feedback in learning and the role of innovative technologies in facilitating note feedback.

Automated Patient Note Grading: Examining Scoring Reliability and Feasibility.

PURPOSE: Scoring postencounter patient notes (PNs) yields significant insights into student performance, but the resource intensity of scoring limits its use. Recent advances in natural language processing (NLP) and machine learning allow application of automated short answer grading (ASAG) for this task. This retrospective study evaluated psychometric characteristics and reliability of an ASAG system for PNs and factors contributing to implementation, including feasibility and case-specific phrase annotation required to tune the system for a new case. METHOD: PNs from standardized patient (SP) cases within a graduation competency exam were used to train the ASAG system, applying a feed-forward neural networks algorithm for scoring. Using faculty phrase-level annotation, 10 PNs per case were required to tune the ASAG system. After tuning, ASAG item-level ratings for 20 notes were compared across ASAG-faculty (4 cases, 80 pairings) and ASAG-nonfaculty (2 cases, 40 pairings). Psychometric characteristics were examined using item analysis and Cronbach's alpha. Inter-rater reliability (IRR) was examined using kappa. RESULTS: ASAG scores demonstrated sufficient variability in differentiating learner PN performance and high IRR between machine and human ratings. Across all items the ASAG-faculty scoring mean kappa was .83 (SE ± .02). The ASAG-nonfaculty pairings kappa was .83 (SE ± .02). The ASAG scoring demonstrated high item discrimination. Internal consistency reliability values at the case level ranged from a Cronbach's alpha of .65 to .77. Faculty time cost to train and supervise nonfaculty raters for 4 cases was approximately $1,856. Faculty cost to tune the ASAG system was approximately $928. CONCLUSIONS: NLP-based automated scoring of PNs demonstrated a high degree of reliability and psychometric confidence for use as learner feedback. The small number of phrase-level annotations required to tune the system to a new case enhances feasibility. ASAG-enabled PN scoring has broad implications for improving feedback in case-based learning contexts in medical education.

The Use of Virtual Standardized Patients for Practice in High Value Care.

INTRODUCTION: This study examined the influence of high value care (HVC)-focused virtual standardized patients (VSPs) on learner attitudes toward cost-conscious care (CCC), performance on subsequent standardized patient (SP) encounters, and the correlation of VSP performance with educational outcomes. METHOD: After didactic sessions on HVC, third-year medical students participated in a randomized crossover design of simulation modalities consisting of 4 VSPs and 3 SPs. Surveys of attitudes toward CCC were administered before didactics and after the first simulation method. Performance markers included automated VSP grading and, for SP cases, faculty-graded observational checklists and patient notes. Performance was compared between modalities using t tests and analysis of variance and then correlated with US Medical Licensing Examination performance. RESULTS: Sixty-six students participated (VSP first: n = 37; SP-first: n = 29). Attitudes toward CCC significantly improved after training (Cohen d = 0.35, P = 0.043), regardless of modality. Simulation order did not impact learner performance for SP encounters. Learners randomized to VSP first performed significantly better within VSP cases for interview (Cohen d = 0.55, P = 0.001) and treatment (Cohen d = 0.50, P = 0.043). The HVC component of learner performance on the SP simulations significantly correlated with US Medical Licensing Examination step 1 ( r = 0.26, P = 0.038) and step 2 clinical knowledge ( r = 0.33, P = 0.031). CONCLUSIONS: High value care didactics combined with either VSPs or SPs positively influenced attitudes toward CCC. The ability to detect an impact of VSPs on learner SP performance was limited by content specificity and sample size.

In Situ Simulation for Adoption of New Technology to Improve Sepsis Care in Rural Emergency Departments.

OBJECTIVES: The aims of the study were to evaluate whether in situ (on-site) simulation training is associated with increased telemedicine use for patients presenting to rural emergency departments (EDs) with severe sepsis and septic shock and to evaluate the association between simulation training and telehealth with acute sepsis bundle (SEP-1) compliance and mortality. METHODS: This was a quasi-experimental study of patients presenting to 2 rural EDs with severe sepsis and/or septic shock before and after rollout of in situ simulation training that included education on sepsis management and the use of telehealth. Unadjusted and adjusted analyses were conducted to describe the association of simulation training with sepsis process of care markers and with mortality. RESULTS: The study included 1753 patients, from 2 rural EDs, 629 presented before training and 1124 presented after training. There were no differences in patient characteristics between the 2 groups. Compliance with several SEP-1 bundle components improved after training: antibiotics within 3 hours, intravenous fluid administration, repeat lactic acid assessment, and vasopressor administration. The use of telemedicine increased from 2% to 5% after training. Use of telemedicine was associated with increases in repeat lactic acid assessment and reassessment for septic shock. We did not demonstrate an improvement in mortality across either of the 2 group comparisons. CONCLUSIONS: We demonstrate an association between simulation and improved care delivery. Implementing an in situ simulation curriculum in rural EDs was associated with a small increase in the use of telemedicine and improvements in sepsis process of care markers but did not demonstrate improvement in mortality. The small increase in telemedicine limited conclusions on its impact.

A novel in situ simulation framework for introduction of a new technology: the 3-Act-3-Debrief model.

BACKGROUND: New technologies for clinical staff are typically introduced via an "in-service" that focuses on knowledge and technical skill. Successful adoption of new healthcare technologies is influenced by multiple other factors as described by the Consolidated Framework in Implementation Research (CFIR). A simulation-based introduction to new technologies provides opportunity to intentionally address specific factors that influence adoption. METHODS: The new technology proposed for adoption was a telehealth cart that provided direct video communication with electronic intensive care unit (eICU) staff for a rural Emergency Department (ED). A novel 3-Act-3-Debrief in situ simulation structure was created to target predictive constructs from the CFIR and connect debriefing to specific workflows. The structure and content of the simulation in relation to the framework is described. Participants completed surveys pre-simulation/post-simulation to measure change in their readiness to adopt the new technology. RESULTS: The scenario was designed and pilot tested before implementation at two rural EDs. There were 60 interprofessional participants across the 2 sites, with 58 pre-simulation and 59 post-simulation surveys completed. The post-simulation mean ratings for each readiness measure (feasibility, quality, resource availability, role clarity, staff receptiveness, and tech usability) increased significantly as a result of the simulation experience. CONCLUSIONS: A novel 3-stage simulation-debriefing structure positively targets factors influencing the adoption of new healthcare technologies.

Sepsis Alerts in Emergency Departments: A Systematic Review of Accuracy and Quality Measure Impact.

INTRODUCTION: For early detection of sepsis, automated systems within the electronic health record have evolved to alert emergency department (ED) personnel to the possibility of sepsis, and in some cases link them to suggested care pathways. We conducted a systematic review of automated sepsis-alert detection systems in the ED. METHODS: We searched multiple health literature databases from the earliest available dates to August 2018. Articles were screened based on abstract, again via manuscript, and further narrowed with set inclusion criteria: 1) adult patients in the ED diagnosed with sepsis, severe sepsis, or septic shock; 2) an electronic system that alerts a healthcare provider of sepsis in real or near-real time; and 3) measures of diagnostic accuracy or quality of sepsis alerts. The final, detailed review was guided by QUADAS-2 and GRADE criteria. We tracked all articles using an online tool (Covidence), and the review was registered with PROSPERO registry of reviews. A two-author consensus was reached at the article choice stage and final review stage. Due to the variation in alert criteria and methods of sepsis diagnosis confirmation, the data were not combined for meta-analysis. RESULTS: We screened 693 articles by title and abstract and 20 by full text; we then selected 10 for the study. The articles were published between 2009-2018. Two studies had algorithm-based alert systems, while eight had rule-based alert systems. All systems used different criteria based on systemic inflammatory response syndrome (SIRS) to define sepsis. Sensitivities ranged from 10-100%, specificities from 78-99%, and positive predictive value from 5.8-54%. Negative predictive value was consistently high at 99-100%. Studies showed some evidence for improved process-of-care markers, including improved time to antibiotics. Length of stay improved in two studies. One low quality study showed improved mortality. CONCLUSION: The limited evidence available suggests that sepsis alerts in the ED setting can be set to high sensitivity. No high-quality studies showed a difference in mortality, but evidence exists for improvements in process of care. Significant further work is needed to understand the consequences of alert fatigue and sensitivity set points.

Impact of an Electronic App on Resident Responses to Simulated In-Flight Medical Emergencies: Randomized Controlled Trial.

BACKGROUND: Health care providers are often called to respond to in-flight medical emergencies, but lack familiarity with expected supplies, interventions, and ground medical control support. OBJECTIVE: The objective of this study was to determine whether a mobile phone app (airRx) improves responses to simulated in-flight medical emergencies. METHODS: This was a randomized study of volunteer, nonemergency resident physician participants who managed simulated in-flight medical emergencies with or without the app. Simulations took place in a mock-up cabin in the simulation center. Standardized participants played the patient, family member, and flight attendant roles. Live, nonblinded rating was used with occasional video review for data clarification. Participants participated in two simulated in-flight medical emergencies (shortness of breath and syncope) and were evaluated with checklists and global rating scales (GRS). Checklist item success rates, key critical action times, GRS, and pre-post simulation confidence in managing in-flight medical emergencies were compared. RESULTS: There were 29 participants in each arm (app vs control; N=58) of the study. Mean percentages of completed checklist items for the app versus control groups were mean 56.1 (SD 10.3) versus mean 49.4 (SD 7.4) for shortness of breath (P=.001) and mean 58 (SD 8.1) versus mean 49.8 (SD 7.0) for syncope (P<.001). The GRS improved with the app for the syncope case (mean 3.14, SD 0.89 versus control mean 2.6, SD 0.97; P=.003), but not the shortness of breath case (mean 2.90, SD 0.97 versus control mean 2.81, SD 0.80; P=.43). For timed checklist items, the app group contacted ground support faster for both cases, but the control group was faster to complete vitals and basic exam. Both groups indicated higher confidence in their postsimulation surveys, but the app group demonstrated a greater increase in this measure. CONCLUSIONS: Use of the airRx app prompted some actions, but delayed others. Simulated performance and feedback suggest the app is a useful adjunct for managing in-flight medical emergencies.

Virtual Standardized Patient Simulation: Case Development and Pilot Application to High-Value Care.

INTRODUCTION: High-value care (HVC) suggests that good history taking and physical examination should lead to risk stratification that drives the use or withholding of diagnostic testing. This study describes the development of a series of virtual standardized patient (VSP) cases and provides preliminary evidence that supports their ability to provide experiential learning in HVC. METHODS: This pilot study used VSPs, or natural language processing-based patient avatars, within the USC Standard Patient platform. Faculty consensus was used to develop the cases, including the optimal diagnostic testing strategies, treatment options, and scored content areas. First-year resident physician learners experienced two 90-minute didactic sessions before completing the cases in a computer laboratory, using typed text to interview the avatar for history taking, then completing physical examination, differential diagnosis, diagnostic testing, and treatment modules for each case. Learners chose a primary and 2 alternative "possible" diagnoses from a list of 6 to 7 choices, diagnostic testing options from an extensive list, and treatments from a brief list ranging from 6 to 9 choices. For the history-taking module, both faculty and the platform scored the learners, and faculty assessed the appropriateness of avatar responses. Four randomly selected learner-avatar interview transcripts for each case were double rated by faculty for interrater reliability calculations. Intraclass correlations were calculated for interrater reliability, and Spearman ρ was used to determine the correlation between the platform and faculty ranking of learners' history-taking scores. RESULTS: Eight VSP cases were experienced by 14 learners. Investigators reviewed 112 transcripts (4646 learner query-avatar responses). Interrater reliability means were 0.87 for learner query scoring and 0.83 for avatar response. Mean learner success for history taking was scored by the faculty at 57% and by the platform at 51% (ρ correlation of learner rankings = 0.80, P = 0.02). The mean avatar appropriate response rate was 85.6% for all cases. Learners chose the correct diagnosis within their 3 choices 82% of the time, ordered a median (interquartile range) of 2 (2) unnecessary tests and completed 56% of optimal treatments. CONCLUSIONS: Our avatar appropriate response rate was similar to past work using similar platforms. The simulations give detailed insights into the thoroughness of learner history taking and testing choices and with further refinement should support learning in HVC.

A Simple Low-Cost Method to Integrate Telehealth Interprofessional Team Members During In Situ Simulation.

INTRODUCTION: With the growth of telehealth, simulation personnel will be called upon to support training that integrates these new technologies and processes. We sought to integrate remote telehealth electronic intensive care unit (eICU) personnel into in situ simulations with rural emergency department (ED) care teams. We describe how we overcame technical challenges of creating shared awareness of the patient's condition and the care team's progress among those executing the simulation, the care team, and the eICU. METHODS: The objective of the simulations was to introduce telehealth technology and new processes of engaging the eICU via telehealth during sepsis care in 2 rural EDs. Scenario development included experts in sepsis, telehealth, and emergency medicine. We describe the operational systems challenges, alternatives considered, and solutions used. Participants completed surveys on self-confidence presimulation/postsimulation in using telehealth and in managing patients with sepsis (1-10 Likert scale, with 10 "completely confident"). Pre-post responses were compared by two-tailed paired t test. RESULTS: We successfully engaged the staff of two EDs: 42 nurses, 9 physicians or advanced practice providers, and 9 technicians (N = 60). We used a shared in situ simulation clinical actions observational checklist, created within an off-the-shelf survey software program, completed during the simulations by an on-site observer, and shared with the eICU team via teleconferencing software, to message and cue eICU nurse engagement. The eICU nurse also participated in debriefing via the telehealth video system with successful simulation engagement. These solutions avoided interfering with real ED or eICU operations. The postsimulation mean ± SD ratings of confidence using telehealth increased from 5.3 ± 2.9 to 8.9 ± 1.1 (Δ3.5, P < 0.05) and in managing patients with sepsis increased from 7.1 ± 2.5 to 8.9 ± 1.1 (Δ1.8, P < 0.05). CONCLUSIONS: We created shared awareness between remote eICU personnel and in situ simulations in rural EDs via a low-cost method using survey software combined with teleconferencing methods.

The 2017 Academic Emergency Medicine Consensus Conference: Catalyzing System Change Through Healthcare Simulation: Systems, Competency, and Outcomes.

Over the past decade, emergency medicine (EM) took a lead role in healthcare simulation in part due to its demands for successful interprofessional and multidisciplinary collaboration, along with educational needs in a diverse array of cognitive and procedural skills. Simulation-based methodologies have the capacity to support training and research platforms that model micro-, meso-, and macrosystems of healthcare. To fully capitalize on the potential of simulation-based research to improve emergency healthcare delivery will require the application of rigorous methods from engineering, social science, and basic science disciplines. The Academic Emergency Medicine (AEM) Consensus Conference "Catalyzing System Change Through Healthcare Simulation: Systems, Competency, and Outcome" was conceived to foster discussion among experts in EM, engineering, and social sciences, focusing on key barriers and opportunities in simulation-based research. This executive summary describes the overall rationale for the conference, conference planning, and consensus-building approaches and outlines the focus of the eight breakout sessions. The consensus outcomes from each breakout session are summarized in proceedings papers published in this issue of Academic Emergency Medicine. Each paper provides an overview of methodologic and knowledge gaps in simulation research and identifies future research targets aimed at improving the safety and quality of healthcare.

Using Virtual Reality Simulation Environments to Assess Competence for Emergency Medicine Learners.

Immersive learning environments that use virtual simulation (VS) technology are increasingly relevant as medical learners train in an environment of restricted clinical training hours and a heightened focus on patient safety. We conducted a consensus process with a breakout group of the 2017 Academic Emergency Medicine Consensus Conference "Catalyzing System Change Through Health Care Simulation: Systems, Competency, and Outcomes." This group examined the current uses of VS in training and assessment, including limitations and challenges in implementing VS into medical education curricula. We discuss the role of virtual environments in formative and summative assessment. Finally, we offer recommended areas of focus for future research examining VS technology for assessment, including high-stakes assessment in medical education. Specifically, we discuss needs for determination of areas of focus for VS training and assessment, development and exploration of virtual platforms, automated feedback within such platforms, and evaluation of effectiveness and validity of VS education.

Advance Care Planning in an Accountable Care Organization Is Associated with Increased Advanced Directive Documentation and Decreased Costs.

BACKGROUND: Advance care planning (ACP) documents patient wishes and increases awareness of palliative care options. OBJECTIVE: To study the association of outpatient ACP with advanced directive documentation, utilization, and costs of care. DESIGN: This was a case-control study of cases with ACP who died matched 1:1 with controls. We used 12 months of data pre-ACP/prematch and predeath. We compared rates of documentation with logit model regression and conducted a difference-in-difference analysis using generalized linear models for utilization and costs. SETTING/SUBJECTS: Medicare beneficiaries attributed to a large rural-suburban-small metro multisite accountable care organization from January 2013 to April 2016, with cross reference to ACP facilitator logs to find cases. MEASUREMENTS: The presence of advance directive forms was verified by chart review. Cost analysis included all utilization and costs billed to Medicare. RESULTS: We matched 325 cases and 325 controls (51.1% female and 48.9% male, mean age 81). 320/325 (98.5%) ACP versus 243/325 (74.8%) of controls had a Healthcare Power of Attorney (odds ratio [OR] 21.6, 95% CI 8.6-54.1) and 172/325(52.9%) ACP versus 145/325 (44.6%) controls had Practitioner Orders for Life Sustaining Treatment (OR 1.40, 95% CI 1.02-1.90) post-ACP/postmatch. Adjusted results showed ACP cases had fewer inpatient admissions (-0.37 admissions, 95% CI -0.66 to -0.08), and inpatient days (-3.66 days, 95% CI -6.23 to -1.09), with no differences in hospice, hospice days, skilled nursing facility use, home health use, 30-day readmissions, or emergency department visits. Adjusted costs were $9,500 lower in the ACP group (95% CI -$16,207 to -$2,793). CONCLUSIONS: ACP increases documentation and was associated with a reduction in overall costs driven primarily by a reduction in inpatient utilization. Our data set was limited by small numbers of minorities and cancer patients.

Cross-Sectional Survey of Physicians on Providing Volunteer Care for In-Flight Medical Events.

BACKGROUND: Airline carriers have equipment, procedures, and protocols in place to handle in-flight medical events (IFMEs). Community physicians may be asked for aid during IFMEs. Cross-Sectional Survey of Physicians on Providing Volunteer Care for In-Flight Medical Events surveyed self-assessed awareness and knowledge, perceived barriers, and suggestions for improving responses to IFMEs. METHODS: We composed a survey regarding clinicians' self-assessed understanding of in-flight resources, procedures, flight environmental issues, and Good Samaritan protections. The survey was distributed primarily via electronic mail to medical staff list serves to a total of approximately 1300 physicians representing 2 health networks that serve urban, suburban, and rural areas in both inpatient and outpatient settings. RESULTS: Total number of responses was 418. Physician response rate was 29.2% (379/1300). In 3% (39/1300), the responder either failed to indicate their background or was another type of health care professional (e.g., dentist, medical student, physician assistant). Of the physicians, 37.5% (142/379) were primary care and 42% (177/418) of responders reported at least one experience of being asked to volunteer. When asked how well they understand the protocols with which medical events are handled, 64% (262/412) responded "not at all" and 23% (94/412) reported "a little" knowledge. Only 56% (223/397) answered that 75% or more of U.S. flights have ground medical support available. There were 73% (298/411) who believed airlines were required to have medical supplies, but 54% (222/410) reported no knowledge of supplies available. A total of 69% (279/403) believed or were sure that the U.S. has a Good Samaritan law that applies to IFMEs. DISCUSSION: Many physicians lack basic knowledge about IFMEs. Responders may assist more effectively if better informed about protocols and the availability of ground medical support. Education and timely information support are recommended.Chatfield E, Bond WF, McCay B, Thibeault C, Alves PM, Squillante M, Timpe J, Cook CJ, Bertino RE. Cross-Sectional Survey of Physicians on Providing Volunteer Care for In-Flight Medical Events. Aerosp Med Hum Perform. 2017; 88(9):876-879.

Implementation of Activity Sensor Equipment in the Homes of Chronic Obstructive Pulmonary Disease Patients.

INTRODUCTION: Telemedicine care models for managing advanced chronic obstructive pulmonary disease (COPD) may benefit from the addition of motion sensing, spirometry, and tablet-based symptom diary tracking. METHODS: We conducted a feasibility study of telemedicine in the home setting using multiple activity sensor monitoring equipment. Deployment and monitoring were supported by home health nurses with technical advice from the equipment makers as needed. Data analytics for motion sensing was provided by the research sponsor, but was not used for care decisions. On study intake, a health risk assessment, Quality of Life (SF-36) survey, and the St. George Respiratory Questionnaire were administered to assess patients' self-perception of quality of life, activities of daily life function, and difficulty living with COPD. RESULTS: Twenty-eight patients were enrolled and data were gathered for a minimum of 6 months and maximum of 9 months. The researchers demonstrated that augmentation of traditional telemedicine methods with motion sensing, spirometry, and symptom diaries appears feasible. The technical, process, logistics barriers, and solutions required for system deployment are described. The researchers demonstrated that augmentation of traditional telemedicine methods with motion sensing, spirometry, and symptom diaries appears feasible. CONCLUSIONS: Further exploration will be needed to determine the value of this information in preventing outcomes relevant to patients.

Deliberate Practice with Standardized Patient Actors and the Development of Formative Feedback for Advance Care Planning Facilitators.

OBJECTIVE: Multimodal curricular assessment after adding standardized patient (SP) actor-based simulation to an advance care planning (ACP) facilitator training course and development of a formative feedback tool. BACKGROUND: ACP represents a highly valued service requiring more and better trained facilitators. METHODS: Participants were primarily nurses and social workers in a large multisite health system. The course included a precourse video demonstration of ACP, traditional lectures, and four 30-minute simulations with SPs. Knowledge was tested with a multiple choice question (MCQ) test. In addition to standard postcourse/postsimulation evaluations, learners were surveyed pre/post/30-90 days delayed for self-perceived confidence. A linear mixed-effects model was used to analyze changes over time. Trained faculty rated performance in simulations with an observational mini-clinical examination (mini-CEX)-type rating form with a checklist, global competency, and global communication rating. Inter-rater reliability (IRR) was calculated on randomly selected paired ratings. RESULTS: Sixty-seven individuals consented to participate. MCQ scores improved from 83% ± 10% to 92% ± 8% (p < 0.001). Paired learner surveys of self-confidence across six domains were available for 65 pre, 65 post, and 40 delayed with a mean positive change on a 0 to 10 point scale from pre-post (2.32 ± 1.65; p < 0.001) and predelayed (2.34 ± 1.96; p < 0.001) time frames. For the faculty observation ratings of simulation performance, the average raw agreement for critical actions was 82% and IRR was 0.71. CONCLUSIONS: Learner feedback and self-assessment suggest that actor-based simulation contributed to improved confidence in conducting ACP. The mini-CEX observation form is adequate for formative feedback, with further testing needed to make judgments of competence.

Validation of a triage algorithm for psychiatric screening (TAPS) for patients with psychiatric chief complaints.

CONTEXT: The process of medical clearance screening for patients with psychiatric chief complaints has not been standardized. OBJECTIVE: To investigate the validity of a triage algorithm for psychiatric screening (TAPS) as a method to screen for the absence of acute medical illness in these patients. METHODS: The current study was a structured, retrospective medical record review in a suburban community teaching hospital with 37,000 emergency department visits per year. All ambulatory patients presenting to triage with a psychiatric chief complaint from January 31, 2001, to June 21, 2002, were assessed with TAPS. Patients with a completed TAPS and a negative assessment were identified and included in the study. A negative TAPS assessment comprised age younger than 65 years, normal vital signs, no medical complaints, no evidence of recent substance use, and no history of schizophrenia, mental retardation, or hallucinations. Emergency department records, return visit records, and inpatient admission records were reviewed for the diagnosis or management of acute medical illness. RESULTS: A total of 1179 patients were assessed with TAPS, of whom 825 (70%) had negative TAPS assessment and were eligible for inclusion. A random sample of 100 patients was selected from this group, with 7 exclusions. Sixty-six (71%) had a history of mental illness and 51 (55%) were admitted. Further, 25 (27%) had laboratory tests ordered, and none of the laboratory results required medical intervention. Twenty-nine patients (31%) received medication, mostly previously prescribed medications or sleep aids. None of the medications were for treating patients with violent or aggressive behavior. The average length of stay was 409 minutes. No patients (95% confidence interval, 0%-3%; P<.05) received a diagnosis of or treatment for acute medical illness. CONCLUSION: The TAPS form is potentially an effective tool in screening for the absence of acute medical illness.

Differential diagnosis generators: an evaluation of currently available computer programs.

BACKGROUND: Differential diagnosis (DDX) generators are computer programs that generate a DDX based on various clinical data. OBJECTIVE: We identified evaluation criteria through consensus, applied these criteria to describe the features of DDX generators, and tested performance using cases from the New England Journal of Medicine (NEJM©) and the Medical Knowledge Self Assessment Program (MKSAP©). METHODS: We first identified evaluation criteria by consensus. Then we performed Google® and Pubmed searches to identify DDX generators. To be included, DDX generators had to do the following: generate a list of potential diagnoses rather than text or article references; rank or indicate critical diagnoses that need to be considered or eliminated; accept at least two signs, symptoms or disease characteristics; provide the ability to compare the clinical presentations of diagnoses; and provide diagnoses in general medicine. The evaluation criteria were then applied to the included DDX generators. Lastly, the performance of the DDX generators was tested with findings from 20 test cases. Each case performance was scored one through five, with a score of five indicating presence of the exact diagnosis. Mean scores and confidence intervals were calculated. KEY RESULTS: Twenty three programs were initially identified and four met the inclusion criteria. These four programs were evaluated using the consensus criteria, which included the following: input method; mobile access; filtering and refinement; lab values, medications, and geography as diagnostic factors; evidence based medicine (EBM) content; references; and drug information content source. The mean scores (95% Confidence Interval) from performance testing on a five-point scale were Isabel© 3.45 (2.53, 4.37), DxPlain® 3.45 (2.63-4.27), Diagnosis Pro® 2.65 (1.75-3.55) and PEPID™ 1.70 (0.71-2.69). The number of exact matches paralleled the mean score finding. CONCLUSIONS: Consensus criteria for DDX generator evaluation were developed. Application of these criteria as well as performance testing supports the use of DxPlain® and Isabel© over the other currently available DDX generators.

Gender disparity in emergency department non-ST elevation myocardial infarction management.

BACKGROUND: Many studies have looked at differences between men and women with acute coronary syndrome. These studies demonstrate that women have worse outcomes, receive fewer invasive interventions, and experience delay in the initiation of established medical therapies. OBJECTIVE: Using innovative technology, we set out to unveil and resolve any gender disparities in the evaluation and treatment of patients presenting with a positive troponin while in the emergency department. Our goal was to assess the feasibility of using a business management query system to create an automated data report that could identify deficiencies in standards of care and be used to improve the quality of treatment we provide our patients. METHODS: Over a 12-month period, key markers for patients with non-ST elevation myocardial infarction (NSTEMI) were tracked (e.g., time to electrocardiogram, door to medications). During this time, educational endeavors were initiated utilizing McKesson's Horizon Business Insight™ (McKesson Information Solutions, Alpharetta, GA) to illustrate gender differences in standard therapy. Subsequently, indicators were evaluated for improvement. RESULTS: Substantial improvements in key indicators for management of NSTEMI were obtained and gender differences minimized where education was provided. CONCLUSION: The integration of these information systems allowed us to create a successful performance improvement tool and, as an added benefit, nearly eliminated the need for manual retrospective chart reviews.

Modeling for the decision process to implement an educational intervention: an example of a central venous catheter insertion course.

INTRODUCTION: The Center for Medicare and Medicaid Services recently declared that central venous catheter-associated bloodstream infections (CLABs) are preventable and no longer reimbursable. The new penalty paradigm creates substantial economic incentives for hospitals to eliminate infections. Modeling exercises offer the opportunity to justify expenditures for the prevention of rare patient safety events. METHODS: We constructed a decision analytic model of the theoretical impact of an educational intervention to improve the safety of central venous catheter insertion. This methodology allows for decision nodes representing procedure urgency, procedure locations, and effects of education on both infectious and mechanical complications. We performed deterministic sensitivity analyses to estimate effects of changes in complication rates, educational impact, and cost. RESULTS: In an already safety system (CLAB rate of 1:1000 line days or 0.7%), the initial sensitivity results suggest that if education results in a 50% reduction in CLAB and a 25% reduction in mechanical complications, survival is equal, and cost is increased by $92,400 in a large hospital system annually. If all CLAB and mechanical complications are eliminated, survival improves slightly, and cost is reduced by $64,800 annually. CONCLUSIONS: These results suggest that if the educational intervention is effective, a small increase in cost can reduce complications. Our analysis does not consider increased revenue generated by virtual bed capacity increases or dynamic changes in practice. This model serves as a template for other health care institutions to estimate the costs and benefits of their own proposed educational interventions.

Competence and retention in performance of the lumbar puncture procedure in a task trainer model.

INTRODUCTION: Our objective was to establish competency and ensure retention in the steps of the lumbar puncture procedure. METHODS: This was a prospective cohort study of first- and second-year emergency medicine residents. Residents completed a survey and then viewed a 5-minute PowerPoint slide presentation and a 15-minute video on performing the procedure. They completed a baseline assessment of competency using a lumbar puncture simulator, received feedback on their performance, and practiced the procedure. They self-recorded the number of practice attempts and performed a second procedure for assessment. Within 3 to 6 months, they performed the procedure for a third observation. The assessments were performed with the same simulator and directly observed by two raters. A previously validated critical actions checklist consisting of 23 critical actions was used. Competency was defined as > or =19 critical actions correct (>80%). Inter-rater reliability was examined using the intraclass correlation coefficient [ICC(2,k)]. RESULTS: Seventeen first-year residents and nine second-year residents completed the initial training. Sixteen first-year residents and eight second-year residents completed the retention assessment. An additional four second-year residents were trained several months into their second year. Twelve of 17 first-year residents and 10 of 13 second-year residents demonstrated competence on the baseline evaluation. All residents demonstrated competence after practice (N = 30) and at the retention check (N = 24). The mean (SD) number of practice attempts before the postpractice assessment was 3.6 (1.1) for first years and 2.4 (2.3) for second years. CONCLUSIONS: This study demonstrated the achievement and retention of competency in the steps of the lumbar puncture procedure in a task trainer model.